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4-H Makes A Difference— Or Does It? 



4-H makes a difference! Ask 4-H'ers seeking membership on a county or state 4-H 
council and they'll quickly tell you how 4-H has changed their lives. Ask the president 
of one of our leading agricultural advertising agencies and he'll proudly share the 
benefits he's received from his 4-H experience— and he wasn't a high achiever in 4-H. 
Talk to members of a community where a 4-H club has just finished a community 
resource development project and they'll say 4-H has made a difference because they 
can see and feel it. 



Ask Deans or Directors of Extension and they'll have a variety of reasons how 4-H has 
made a difference. It may be the legislature has just voted continued or increased 
appropriations for Extension because of 4-H. Ask governmental leaders and they'll tell 
you how important *-H is to their constituents. 

Of course, 4-H makes a difference, b ut can we document it ! Can we show substantive 
research evidence that young people~or adults are different because they participated 
in 4-H. There is not as much research data available as we should have or need. There 
are not many sound scientific research projects. So does 4-H make a difference? 
Frankly, we don't know as much as we need to. 



Research in the 8©'s 

One of the hallmarks of 4-H in the 80's must be research. We can't continue to survive 
on testimonial data to maintain existing support or gain new support. Everyone from a 
county staff member in the smallest county to the federal administrator must have 
research data to use in making decisions related to continuing, adapting or creating 
new 4-H programming for youth in the 1980's. Note I didn't project this need, as 
sometimes we do, as a need for the future. We needed it last year and this year and 
even more as each year passes. 

We need two types of research data. First, we need impact data to demonstrate 
results of our educational programming. Second, we need research data to use in 
improving our existing methods of working with youth and adults in 4-H and to help us 
design new programs for existing and new audiences. Gone are the days when we can 
afford to design 4-H educational experiences on myths or the ever popular intuition. 
Resources (time, energy, money) are scarce. We don't have the luxury of 
experimenting as we go. We need to base programming on sound research so we can 
reduce risks and increase efficiency and effectiveness. 



Impact {Studies 

Impact studies need to grow out of program priorities and therefore, a state by state 
listing of research priorities is more appropriate than a national master list of needed 
impact research needed. At the same time, it is important to recognize we need to 
aggregate data on a national basis to document accomplishments in 4-H. As a result 
states as well as the federal partner should identify impact data needed. 



Presented to Symposium on Research Needs for Extension Education at Ohio State 
University, Columbus, OH, May 25, 1983 by 3erry Parsons, State Leader, 4-H and 
Youth Programs, Iowa State University, Ames, IA 50011. 
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My list of impact studies is short because of the way I've chosen to categorize the 
needs. They are from an Iowa perspective but I believe they also reflect some national 
concerns and needs. Pm suggesting the following impact studies: 



1. Life Skills 

We need to develop a series of studies to document the life skills learned by 
4-H'ers who participate in a specific event, activity, or project. There is a 
real need to know if youth who participate in a livestock project have 
learned life skills or when young people go to a State Leadership Workshop 
what life skills they have learned. One of the criteria for evaluating events, 
like our State 4-H Conference, should be whether delegates learn some life 
skills or was it just a good outing. At least in Iowa we don't have these 
answers and we need them. 



2. Skills and Knowledge 

In the previous priority, I was talking about life skills but we need to know 
about subject matter skills and knowledge young people learn in 4-H. For 
example, we need to know what 4-H'ers have learned about integrated pest 
management, meal planning, raising livestock, caring for a pet or whatever 
they have studied. 

In recent years 4-H has been criticized by some subject matter specialists 
who say we aren't interested in subject matter skills and knowledge. This is 
not true learning subject matter skills and knowledge is as important as it 
always has been. In my judgment, both are important. One aspect of our 
4-H heritage is that 4-H started as a way to teach canning and corn 
production. We taught children production skills so that could teach their 
parents those skills. In 1980 this concept is still valid. In Iowa I think we 
"missed the boat" with IPM. We should have introduced a 4-H IPM project 
at the same time we introduced IPM for adults. There are numerous other 
examples where youth and adult education can and should be closely 
interrelated. If we aren't careful, we'll do the same with computers. 

We need is to demonstrate through research that young people do learn 
useful subject matter skills and they do teach their parents or other adults. 
(Incidentally, this means we need to be teaching "cutting edge" knowledge to 
youth—not outdated subject matter or skills as we do sometimes.) 



3. Longitudinal Study 

I hesitate to introduce my third priority because it is a traditional practice 
for researchers to identify longitudinal studies as a need. I had it on my list 
and crossed it off, but then finally said, "Yes, we do need it!" I started this 
discussion saying we accept 4-H as making a difference. We believe it 
influences the lives of young people but most often these results are not 
recognized at the end of a project or a camp or when the young person 
ceases to participate in If we could document this growth, we could 

easily command increased fiscal resources and unprecendented increase in 
attitudinal support. Longitudinal studies are the only way to do this but 
they are costly and difficult to manage. Studying matched sample of youth 
involved in b-H with those not in 4-H as they move from childhood into 
adulthood would be invaluable. 



It would be a daring step. Most youth-serving agencies aren't able or willing 
to risk the scrutiny of such a study. After all the results may not be 
positive. 4-H can and should! 



Program Research 

Earlier I alluded to specific ft-H methodology - projects, workshops, community clubs, 
school enrichment, etc. - that enables us to effectively work with young people so that 
they learn life skills and subject matter skills and knowledge. We believe strongly in 
these methodologies and have some evidence that they are successful. In the 1980 f s 
we need more data to sharpen our ability to deliver effective educational programs for 
youth. In fact, my list is longer (it's not so easy to consolidate) and equally important 
as impact studies. The Extension in the 80 f s report clearly states this as a need. I see 
the following as priority research needs in this decade: 



1 . Progression of Experiences 

In recent years we've moved to short term learning experiences - school 
enrichment, special interest - in addition to our more traditional long term 
experiences— community clubs. We have debated and discussed if our short 
term experiences will encourage young people to progress to longer term 
experiences such as community clubs. 

Now is the time to evaluate this progression of learning experiences 
phenomena to see: 

a) is it really happening, 

b) if so, what methods or approaches are more effective, and 

c) how can we become more effective and efficient. 

2. Delivery Modes 

During the last decade 4-H has become a multiphased educational program. 
As such we have used a number of delivery modes— special interest, school 
enrichment, TV, community clubs, project groups, etc. A scientific research 
study is needed to document the strengths, weaknesses, accomplishments, as 
well as successful organizational strategies for each mode so program 
developers can match delivery modes with needs of youth, subject matter, 
etc. 



3. Publications 

After staff salaries and benefits, the next largest expenditure in 4-H is 
publications. Ironically, we know very little about the effectiveness, use or 
format of publications for 4-H'ers. We continue to develop, print and 
distribute publications like we know these answers. We don't! We need to! 

4. Volunteer Staffing 

Volunteer staffing is the backbone of 4-H. Someone from industry who saw 
this recommendation would be aghast and rightly so. We do not have 
research data to demonstrate the most effective and efficient volunteer 
staffing pattern. We have a variety of volunteer staffing patterns in the 
United States, even in a single state, and we need to analyze these patterns 
and based on research data make recommendations for implementation. 
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5. Professional Staffing 

The same can be said for paraprofessional and professional staffing patterns. 
In Iowa we have two major 4-H staffing patterns and at least a dozen 
variations. We have very limited data to document our staffing decisions, 
we need more. This situation is duplicated in most states. 

6. Self Directed Learning v 
Allen Tough is, I believe, the premier adult education researcher. His 
research has been replicated around the world with adults and youth. Yet in 

4-H we only use his data in a limited way. We desperately need to conduct 
applied research to adapt the Tough concept of self directed learning to 
for youth and adults. 

7. Computer Learning 

Lastly, this idea is probably the responsibility of traditional educators but 
why leave the really innovative research to others? As a result I'm 
suggesting Extension researchers get involved. Do we really know how 
computers aid learning? Can we really teach decision-making skills with a 
computer or are we using the computer as a device to attract attention? 
We need to know more about computer assisted learning. In 4-H we have 
many opportunities to be involved in a landmark research effort. 



Strategies for 4-H Research 

Professional 4-H staff members tend to be trained as practitioners. They are skilled in 
designing and implementing programs. Until more recently, they have had limited 
skills and competencies in research. This situation is changing rapidly but at the 
county level we'll never have highly skilled researchers (nor do I believe we should 
have). In addition, the work load of professional 4-H staff is heavy and demanding. 
They have little time for insightful contemplative research thinking. 

With the decentralization of Extension programming, the diverse youth needs and 
multiple delivery methods for teaching youth there still exists a need to aggregate 
research data across county, state, and regional lines. This situation provides a unique 
opportunity for researchers to design ways to facilitate research data collection, 
interpretation and utilization. The pragmatist in me tells me we need to do several 
things to enable research to occur and to help use research data. I suggest: 

1 • Research Linkages 

With a few exceptions, state 4-H departments have no direct linkage to 
academic departments with research components. To compensate, some 
4-H staff groups have hired a research or evaluation specialist. The 
latter staffing pattern I believe is an acceptable approach but may be 
limiting in it's ability to secure and deliver research data to youth 
programming. 

For a long time I have believed at least som* of the state 4-H staff 
members should be directly linked with academic departments. Note, I said 
departments. I believe we need to utilize research from several different 
disciplines or departments. At Iowa State University, these would include 
Child Development, Family Environment, Agricultural Education, Adult ♦ 
Education, Elementary and Secondary Education to name a few. Ncc only do 
we need to have access to research faculty but we need to utilize teaching 
faculty resources to augment the competencies of Extension specialists. 
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Forging 4-H and academic ties is not easy. Academic rank, tenure, staff 
interests and goals and objectives all complicate the situation. The need 
does exist and we should work towards a goal of forming strong linkage 
between 4-H and academic departments. 



2. State Staffing Pattern 

Many of us who are administratively responsible for 4-H need to rethink our 
state staffing pattern so the synthesis of research, creative programming, 
implementation of programs is more efficient and effective. There are at 
least four major state staff functions (when you include the state leader). 
These functional staff positions are: 

a) specialist - collects research data and synthesizes it into useable 
information for implementing programs for youth and adults and 
develops and refines effective youth educational experiences. 

b) field support - assists field staff members with program development 
and implementation, including applied research, liaison with state staff 
and stimulaties of new program implementation. 

c) program assistant - handles routine procedural matters like 
publications, data processing, events and records. 

d) administrative - supervise personnel, provide linkage with other 
program leaders and assists with resource development. 

Currently many states, including Iowa, mix these functions so a staff 
member is expected to do all or at least the first three. As a result staff 
have widely differing responsibilities. In our staff I'm noting frustration and 
a lack of good research development and utilization. Under this staffing 
pattern application of research is probably not done unless we have hired 
specialists who are really interested in research and then they probably 
short-change the other functions or are extremely over-extended in their 
work. 

3. Q.E.E. Approach 

We need to be able to use the Q.E.E. approach to 4-H research— quick, 
efficient and effective. Note I did NOT say "quick and dirty." 

We need to aggregate data but staff are busy and as it said at the field level 
not necessarily highly competent researchers. We need some research 
models that can be easily and quickly used. We need to have the 
methodology spelled out, questions designed (of course, we could modify 
them) microcomputers packages to facilitate data analysis and reporting. 
These models need to be ready so they can quickly and efficiently be applied 
to a problem and we can effective secure research data. 

Wouldn f t it be great to say to a staff members, here is a research package 
you can use to gather data on life skills learned in livestock projects and 
when they are done, they would have usable data for their situation, but at 
the state, regional or national levels we could aggregate these data because 
a standardized methodology was used. 
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It would be a great research project for a doctoral student to design, and 
test a workable model? I f m ready to use it in Iowa— NOW! 

4. Financial Resources 

I can't leave this topic without saying something about financial resources. 
Yes, resources are scarce. Yes, private resources can be used. !f we 
believe evaluation is an integral part of the 4-H program development 
process (at least I do), then we need to allocate some of our present 
resources to research/evaluation before we ask private donors for support. 
It's not fair to ask private funds to do something we ought to be doing and 
aren't willing to do with our own resources. 

SuMMMary 

I'm reminded of the old adage we've often attributed to an old-time, crusty, county 
agent (note I didn't say 4-H staff member) who advises the new Extension staff 
member to not get too excited about the new program from the state office. You 
know the statement— "this too will pass!" When talking about 4-H research I don't 
believe it and furthermore, if we as professional Extension educators "let it pass," we 
will be short-changing our clie r ,*e - 4-H'ers and the adults who work with youth. 

My crystal ball tends to be a bit cloudy— cloudy with my biases— but then I'm not 
speaking for all state leaders or county, state, or federal staff. I believe these are 
some of the pressing research needs and some ways we can improve research data 
collection and use. I hope my thoughts will stimulate researchers to undertake these 
projects or these ideas will stimulate even better research ideas. 

One last thought! M-i/Motherhood/Appie Pie have been labeled as "All-American." It 
is interesting to note the last two have come in for much scrutinizing during the last 
decade and some do* K * have been expressed about our undying loyalty to the old 
concepts we have he about both. Perhaps we need to be concerned about 4-H, too. 
In 4-H we may believe in and practice what just might be a myth or an unproven facts* 
Is it possible that we resist research because we aren't sure of the outcome? 

Of course, 4-H makes a difference! Or does it? Do we really want to know? I for one 
do. And I do believe "4-H Makes A Difference." 
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RESEARCH NEEDS IN THE 
ADMINISTRATION AND SUPERVISION OF COOPERATIVE EXTENSION 



Introduction 

I am most pleased to be with you today. I compliment you at Ohio Scate 
for organizing this useful symposium. 

During the past two years I have been in the role of administering a 
rather large Extension organization* This experience has given me a fairly 
broad perspective of the administration of three major types of Extension 
units : 

Cooperative Extension: 560 faculty/staff with a broad program and a $25 

million budget* 

General Extension: 445 faculty/staff with an extensive program in 

general, liberal, and professional continuing 
education and a 422 million budget ($15 million 
program revenue). 

Telecommunications: 140 faculty/staff and a $3.6 million budget to 

operate statewide radio and television plus 
teleconference and dial access facilities and 
slow-scan TV operations. 

In addition, we have 200 staff and nearly $5 million in administration and 
administrative services to support the programs of the three programming units. 

The University of Wisconsin-Extension is not merely a free-standing 
institution. It is the budgetary and program coordinating mechanism for the 
extension, continuing education, and outreach activities of the 13 
universities and 12 centers of the University of Wisconsin System. In the 
past year, we have carried out an extensive internal reorganization to 
simplify our relationships with the other 13 universities and the 12 two-year 
center campuses of the UW System. And, we are in the midst of fulfilling a UW 
Board of Regents policy to "integrate" all extension programs with the 
departments, schools, and colleges of the UW universities and centers. The 
agricultural and home economics programs and several other Cooperative 
Extension programs have been integrated with the University departments for 
many years ... but the current challenge is to integrate the extension 
faculty/sfcaf f of all the continuing education programs in such areas as 
health, medicine, engineering, and business. 



Presentation by Acting Chancellor Patrick G. Boyle, University of 
Wisconsin-Extension, May 25, 1983, Symposium on Research Needs for Extension 
Education, Ohio State University, Columbus, Ohio. 
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Many people believe that the administrator of such a large and complex 
educational institution would spend most of his or her time dealing with 
important issues of budget, program, and personnel* This is true • ♦. a large 
share of my time is devoted to resolving concerns over these important 
questions. However, a large share of my time also is spent in other ways* As 
examples, I'd like to list a few of the concerns that demanded my attention 
this week: 

• A draft statement from the faculty senate on collective bargaining* 

• A letter from an irate legislator demanding that more extension 
faculty and budget resources be located on a two-year center campus 
in his district. 

• An insurance claim for a small fire in one of our conference centers. 

• A question from the facilities manager over what to do with some 
surplus chairs and desks stored in an office he wanted to assign to a 
new employee. 

• A survey request from a UW System committee studying the archives 
policies of the UW institutions ••• and another request to appoint an 
Extension archives committee because we realized that we didn't have 
any archiving policies. 

• A question from the President's Office on what Extension personnel 
were planning to attend a retirement party for a state legislator. 

• A request for my participation in a workshop on sexual harassment. 

• Approval of some vacation request forms from Chancellor's office 
staff members. 

• A request for a child care center for extension employees. 

• A decision over what color carpeting to use in remodeling the 
Chancellor's conference room* 

• A donation for flowers for a staff member with a death in the family. 

All this goes to show is that there's much more to being the administrator 
of a large institution of higher education than meets the eye. And all these 
small concerns are important insofar as they contribute to building staff 
morale, 4 aspiring trust, and letting people know that the administrator is a 
human . ~ing who cares about their personal concerns, as well as about the big 
issues that affect the entire institution* 

The experience of the past couple of years has caused me to put the 
internal management of a CES in perspective when trying to relate the 
University of Wisconsin-Extension to the University of Wisconsin System 
Administration, the Board of Regents, the chancellors of the other UW System 
institutions, state government, and many specific interest groups and 
constituenc ies* 
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So my observations about the research needs of the administration of 
Cooperative Extension are derived not only from several years within 
Cooperative Extension, but also from this broader perspective of a very 
comprehensive Extension program. 

t Cooperative Extension Administration and Supervision 

The job of an administrator in Cooperative Extension is not an easy one. 
A number of factors, separately and in relationship, have emerged over the 
past decade or so which have impacted greatly on the roles of Cooperative 
Extension administration* These include such things as: 

1. The complexities of the university today and the pressure demands on 
top university administrators who are frequently unfamiliar, 
inexperienced, or disinterested in programs such as Cooperative 
Extension. 

2. Social and economic change - the increasing public and government 
focus on human rights, equal employment opportunities, and 
affirmative action. The sorry economic situation of the past several 
years has led to a strong public and government focus on the 
responsibility of educational institutions to re-educate workers, to 
aid in economic recovery, to assist business people, and to provide 
more economic impacts for the investments of tax dollars in education. 

3. The increasing number of agencies and institutions entering and 
desiring to enter the educational and informational fields Extension 
engages in. For example, large corporations, such as Wang and IBM, 
are now offering continuing education and advanced training programs 
for their employees, to fill the gaps in advanced technology that 
institutions of higher education simply aren't ready to fill yet. 
And, with more mandated professional education requirements, 
professional associations are conducting their own continuing 
education programs. We also have competition Urom state agencies, 
vocational schools, civic groups, private clubs, private schools and 
colleges. 

4. The move of society, generally, and educational institutions, 
specifically, toward participant inputs into decision making, into 
all decisions affecting their well-being, including unionization and 
standardization. 

5. The emergence of numerous "special interest 11 groups and organizations 
which have entered the lobbying field and are influencing decision 
making at the community, state and national levels. 

* Thus, the Extension administrator who once spent a large proportion of 

time relating directly to the program priorities and personal management of 
the extension operations no longer has time to be personally acquainted with 
all of the staff, nor to deal with many of the key program and policy issues 
directly. Rather, the administrator must devote his/her time and many 
additional financial resources to concerns of these special interest groups; 
to interagency and interinstitutional competition and cooperation; to the 
decision making power structure of the counties, the state and the federal 
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government; to lawsuits or threatened individual personnel actions of various 
types; and to "educating 11 university administrators, regents, and state 
executive officials about the role and value of Extension programs. 

The social changes also have affected the roles of other types of 
Extension administrators, who must devote more of their time to 
accountability, external relationships, and responding to concerns that are 
beyond the immediate scope of their programs. The societal factors I outlined 
have contributed significantly to the difficulties and complexities of today's 
Cooperative Extension administrators. I believe they have also contributed to 
the research needs in Cooperative Extension administration and supervision. 

To establish a framework for our discussion of research needs, let me 
describe my interpretation of administration and of research. Administration 
involves both an art and a science, particularly when we think of art as being 
skillful in working with people; providing them an opportunity to utilize 
their talents to the maximum; providing them an operational situation which 
continues to challenge their initiative; and providing that type of counsel 
and support which instills in them an appreciation of being a member of an 
efficiently functioning team. 

There also is an ever-deepening appreciation of the fact that extension 
administration needs to know more about the science of administration. Skill 
in performance is always improved when that performance is based upon facts 
and tested with proven principles. We have many more tools now at our 
disposal, such as computer information systems. Th challenge is for 
administrators to learn how to use them effectively. 

This leads me to my favorite definition of an administrator's role. 
According to Daniel Griffiths: 

An administrator directs and controls the decision-making process. If the 
administrator is the controller of the decision-making process, rather 
than the maker of organizational decisions, the decisions will be more 
effective. 

Another old and very simple definition of administration is: 

"The guidance, leadership, and control of the efforts of a group of 
individuals toward some common goal. 11 

Perhaps we also need to define research. "Research is not the mere 
collection of facts which are infinitely numerous and most uninteresting but 
the attempt for the human mind to order these facts into satisfying patterns." 

Research Needs for CES Administration 

What are the important research needs for the administration and 
supervision of Cooperative Extension today? As you might guess, these needs 
are more complex and far-reaching than they once were, in the days when 
education, like everything else, was simpler. 

I've categorized the needs for research in Cooperative Extension 
administration into nine general areas. In each, I've noted some questions 
that need research answers to be helpful to Cooperative Extension 
administration. I'm sure the reactor panel will be able to think of more: 
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1. The first area for research is the Image of Cooperative Extension . This 
area refers to the perception that the formal and informal leadership of a 
state has of CES. An organization's image determines to a great extent 
its influence and effectiveness. Image includes publicity, the public 
presence of Extension and the idea that people have of the organization. 
With regard to public decision-makers, we can ask two questions: 

A. What criteria do decision-makers use to judge the value of Extension? 

B. What factors contribute to a favorable/unfavorable image of Extension 
or causes decision-makers to be impressed and allocate more resources? 

With regard to the general public, the audiences or participants in 
extension programs, we can also ask some research questions: 

C. What factors influence members of the general public to be, impressed 
with extension programs and want to participate in them? 

D. How does the image of extension influence participation by more urban 
people, as farm and rural populations decline? 

2. The second area for research is Administrative Climate . Administrative 
climate referr. to the whole atmosphere generated within an organization by 
those persons in administration. A favorable ego-enhancing administrative 
climate helps to give employees a sense of acceptance, of security, of 
having ready access to superiors. It adds to the faculty's contentment, 
productivity, enthusiasm, and sense of well-being. 

A. What is a productive and effective administrative climate? 

B. What conditions need to exist for a productive climate? 

C. What factors or conditions cause the administrative climate to be 
either positive or negative? 

D. How does an administrator's personal style influence his/her 
effect iveness? 

E. Are Cooperative Extension administrators more autocratic or 
democratic and is either style a cause for success or a detriment to 
Extension programs? 

3. A third area for research is Decision-Making Within the Organization . 
Contemporary theories of decision-making and reasoning have considerably 
strengthened our potential for effective action. The challenge is to 
effectively and consistently implement a decision-making model within the 
organization. 

A. What is participatory decision-making? On what basis is it effective? 

B. What processes should the administrator use in making decisions 
across program areas? 



C. What hierarchical structure is most conducive to establishing formal 
communication channels and relationships for decision-making? 



D. Can the Extension administrative process be made more effective by 
greater adherence to either formal or informal organizational 
concepts? 

4* Fourth on my list of research need categories, and related to the 
administrative climate, are studies of Organizational Behavior . 
Organizational behavior is a broad concept referring to inter-personal 
relations, group actions, interactions and human relationships of all 
kinds, both formal and informal, within the organization. It includes the 
group factors that contribute to organizational performance and 
effectiveness* It pertains also to those employee attitudes and reactions 
to management that affect productivity. Organizational behavior should be 
looked at from both a macroscopic and a microscopic point of view. 

A* What factors contribute to effective interactions and communications 
among a CES faculty? 

B. How can the authoritarian or overly permissive behavior of a district 
supervisor be controlled? Should it be? 

C* Are professional development experiences in inter-personal relations 
effective? Under what conditions? 

5* A fifth area of research need concerns Personnel Management and Collective 
Bargaining . 

The critical tasks of recruitment, selection, orientation, training, and 
performance appraisal are familiar to all of us. The research needs relative 
to these unanswered questions are numerous. Collective bargaining is a major 
new dimension of personnel management. 

A* What doos a new employee really need to know about Extension, program 
development, etc., during the first year on the job? 

B. How does collective bargaining affect the middle manager's role? 

C» Does collective bargaining affect participatory decision-making or 
the decision-making structure of CES? 

D. Does collective bargaining improve the educational program? How? 

6» A sixth category for research deals with Supervisory Approaches and Middl e 



The concept of supervisory approaches encompasses the arrangements an 
organization makes for appraising performance, staffing, training, and 
implementing the reward system. This is an area within administration that 
has to do with management's efforts to have employees work together toward the 
objectives of the organization. 



Managers . 




A. Under what conditions is counseling more effective than advising? Is 
advising ever desirable f.n a supervisor-agent relationship? 

B. What criteria should be used to evaluate the effectiveness of middle 
managers (program Leaders/district directors)? 

C. What characteristics of middle managers influence the satisfaction 
and productivity of employees? 

7. A seventh area for research is Resource Allocation and Fiscal Management . 

Resource allocation includes the legal and administrative requirements, 
accounting policies and assignment of accounting responsibilities. Allocation 
of resources involves the question of relative values. It's not the challenge 
of deciding whether a particular activity is worthwhile, but of whether it is 
more worthwhile than other things on which the funds could be spent. 

A. In a generic sense when or how does administration judge the optimal 
allocations to a program or an area of need; e.g., agriculture versus 
home economics; evaluation versus long-range planning; personnel 
versus capital equipment? 

B. What would be the effect of expanding the resources available by 
charging student fees for selected Cooperative Extension programs? 

C. How can reallocations best be handled as one program is judged to 
have met a need while another is emerging as a high priority? 

8. An eighth area of needed research is Constituent Relationships and the 
Political Arena . 

In these days of heavy demands for accountability, budget cutting of what 
are seen as "non-essential 11 programs, tight public funding, and large numbers 
of citizen and private lobbying groups on all sorts of special interests, 
constituent relationships and political know-how have emerged as high priority 
needs for Extension administrators. 

A. What types of group efforts are most effective in gaining the 
favorable attention of state legislators, county supervisors, and 
other decision-makers? 

B. How can a program such as Cooperative Extension, with an historically 
rural image, gain acceptance as a positive benefit by a large number 
of powerful urban legislators? Does it need to? 

C. What are the professional development needs of CES administrators for 
them to become knowledgeable and effective in the political arena? 

9* My ninth suggested area for research has to do with the mission and 

priorities of the Cooperative Extension Service . Cooperative Extension 
has changed with the times in many cases. In some states, however, 
traditional agricultural extension work has remained the priority since 
the CES was founded. Other states have moved on to deal with emerging 
concerns, such as energy, small business development, environmental 
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resource development, groundwater quality, and many other areas. This 
research area concerns the emerging needs of people and communities as the 
numbers of people involved in agriculture, homemaking — and even the 
numbers of young people available for youth development programs — 
decline in the future. What can we as administrators do to ensure that 
our programs do not become obsolete? What are our CES priorities for the 
future and ho * do we arrive at them? 

A. What internal decision-making and long-range planning processes will 
be effective to define the future CES mission and program priorities? 

B. Who will be involved in making decisions on future needs and 
prioritie s? 

These are only a few of innumerable possible areas for research in the 
administration and supervision of as complex an organization as Cooperative 
Extension. I'm happy to be able to share my ideas with you today and am eager 
to hear the interpretations and ideas of the reactor panel on these important 
areas of need. When questions such as these arc answered, with a scientific 
body of data to back up the conclusions, the administration of CES will be in 
a better position to face the future. 
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Research Needs in Extension Personnel Development 

I. Introduction 

The organization and management of Extension personnel has been a 
topic of discussion since the inception of Cooperative Extension work in 
1914. Historical review suggests that as soon as the Extension Services 
in various states became larger than a single director in a university 
office, processes and functions of the organization emphasized improving 
the quality and quantity of Extension staff at every level. Early reports 
about district conferences and staff meetings identify personnel development 
as a significant goal. We, in Cooperative Extension, have been abouc the 
business of personnel development for a long time. Depending upon who 
you talk to however, we have succeeded beyond our fondest dreams or 
failed miserably in developing the staff which comprise Cooperative Extension 
Services across the United States. 

Personnel development is a complex process. That complexity is apparent 
as we attempt to define the process. To me, personnel development is 
all of the procedures and functions that affect a staff member in an 
organization while he or she is attempting to accomplish the goals 
encompassed in the work environment. This definition goes far beyond 
what is commonly conceived of as staff development and includes the personnel 
practices of hiring, selection, orientation, performance appraisal and 
feedback so important to the development and growth of Extension faculty 
members. I have conceptualized this process in a model (Figure 1) which we 
have used with some success at the University of Minnesota. This model 
suggests that personnel development actually begins when a vacancy occurs 
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and includes a number of processes which are cyclically repeated throughout 
an individuals career. Processes like program planning, specific performance 
expectations, staff development, performance development and review, and 
performance evaluation occur annually in roost Extension organizations. 
All of these impact on an individual in their work environment. 

One of the problems in many organizations is that these functions 
are not addressed as a comprehensive whole. Program planning is accomplished 
without regard to performance evaluation. Staff development is an 
autonomous function many times not related to specific performance 
expectations. The processes of performance development and review or 
performance evaluation are accomplished in order to assign a merit salary 
or academic promotion rather than as functions related to improving work 
performance of the staff member. We are too often intent on improving 
"our" processes while expecting the Extension faculty member to bridge 
the gaps between these functions. 

An example of this diversity is provided by an examination of where 
the personnel and staff development functions originate. Their separate roots 
in most organizations suggest to Extension administrators and faculty members 
little relationship between personnel practices and staff development 
opportunities. It is my thesis that these functions must be related and 
integrated if their impact is going to have a logical or sensible effect 
on the individual Extension worker. If our concern is to improve the 
capabilities of the Extension staff member, close coordination of these 
processes is necessary. 
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II. Research Needs 

Much research must be accomplished to better understand not only the 
relationships between but also within each of the processes identified 
on the personnel development model as I have conceptualized it. I will 
attempt to identify some possible research topics in each of these 
functional areas recognizing that no collective survey has been accomplished. 
In fact, I would suggest that an important initial research project would 
be testing the impact of this personnel development model on a Cooperative 
Extension organization. 

Search Process 

In any typical year at the University of Minnesota Agricultural 
Extension Service we search for about 50 staff members. Consequently, 
there are a number of search processes operating simultaneously. Yet we 
know very little about what constitutes a productive search. In some cases 
a number of candidates apply. In other cases something goes awry so that 
few appropriate candidates indicate an interest in the position. The impact 
of equal employment opportunity on the search process is not known. 
Profitable methods of search and recruitment have not been tested. How 
Extension Services communicate with potential applicants through the search 
process is uncertain. Typically -;e do things the way we have always 
searched unless forced by some new federal legislation or university 
court order to alter our process. 

Selection 

Once the applications arrive we make judgments about the best potential 
Extension faculty members. The National Study on Personnel Selection gave 
us some ground rules and guidelines for selection. The national study 
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may have provided a basis but certainly a major finding of the study was a 
non-finding. Very few of the procedures we have used in a selection 



process seem related to success on the part of the county Extension worker* 
For instance, we do not know if there is a relationship of various 
academic backgrounds to successful performance and length of service as a 
county Extension agent. We truly do not know the key factors chat seem 
to generate successful Extension work. 

Orientation/Induction 

Orientation is the process whereby a new staff member finds out about 
the organization that employed them. Induction provides information to the 
new person about his or her specific role in that organization. Extension 
administrators have long been concerned about this process. The concern 
generally has been based on where the processes are effective and provide 
ease of entry into the organization. Extension needs a comparative study 
of orientation and induction processes with a measure of resultant attitudes, 
commitments, and performance in an attempt to identify some relationship 
of the processes applied to the performance and satisfaction of the new staff 
member. We need to know more about how educational backgrounds ought to 
influence the orientation or induction process. We also need to determine 
with more precision what an oriented or inducted staff member looks like. 
In the meantime we go merrily on our way applying band-aids when in fact 
major surgery may be required. 

Program Planning and Goals 

Much has been written about research in planning Extension programs. 
The relationship however, between program planning and personnel development 
has not been as clearly articulated in the literature. A common attitude 
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held by Extension staff is seeing the program planning process as "something 
done for administration" rather than understanding it as a basic process 
for improving performance, results, and personal growth. 

Specifi c Performance Expectations 

Specific performance expectations are usually assumed to be communicated 
through revised position descriptions. However, most position descriptions 
I have seen are generic and encompass the world rather than really 
providing guidelines for an individual's performance. When questioned, 
supervisors suggest they want to provide the individual job freedom and 
individuals suggest they do not want anyone "backing them into a corner." 
In fact, the lack of precision in specific performance expectations often 
fails to set direction for either improved performance or personal growth. 

We do know from research that role dissonance is a major source of 
dissatisfaction and reason for employee turnover. What we lack are 
models for effectively determining specific performance expectations 
and processes for modifying and changing those expectations as the 
organization and/or the individual changes. We need to explore the impact 
of specific performance expectations on actual performance. 

Staff Development 

Inservice education opportunities are often provided for Extension staff 
by our organizations. In Minnesota we provide inservice education where 
the individual selects the kinds of opportunities they feel best suited to 
their personal growth and inservice training where the organization identifies 
the type of educational opportunities needed by a staff member and requires 
participation. We do this based on the feeling that it is important for 
continuing educators to continue their education. But, we are not certain 



9 

ERLC 



26 



of the relationship of these educational interventions to improved 
employee performance, increased commitment, or satisfaction. What factors 
are most affected by staff development? Does it do what we want it to do? 
We also ponder whether participation in staff development generates movement 
out of the Extension Service rather than more effective performance. 

We conduct a number of other staff development opportunities without 
the benefit of any understanding of the preferred learning style of 
county Extension agents. It often appears that we are more concerned 
about teaching than learning. We need to investigate what causes change 
in behavior of Extension faculty most effectively and efficiently. 
Incidently, learning more about how our faculty learn and change might 
give us additional insight/answers into ho / our clientele learn and change. 

With regard to methodologies, we deliver staff development the same 
way we do Extension programming for our clientele. We are not sure about 
different delivery systems nor do we place a high priority on learning 
and whether computer based staff development, correspondence courses, radio 
or video tapes really provide learning for our staff as well as or better 
than the traditional workshop or classroom setting. 

Too ofteu j i staff development we assume quantity is quality. 
Possibly a better understanding of the individual faculty member, their 
work environment, and a preferred learning style would help us eliminate 
the shackles of staff development tradition and bring us closer to true 
personnel development. 

Performance Development and Review 

Performance development and review is a part of the performance 
appraisal process** It is an interaction whereby an individual serves as 
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a helper in providing feedback to an individual on their performance 
and suggesting ways to encourage personnel development* No judgment or 
scores are offered but rather this is an opportunity to problem solve 
with the individual. How important are these interactions to the 
performance and success of an individual Extension worker? tfhat is 
the proper methodology for doing performance development and review? 
Can a supervisor truly do performance review as a helper or must it be an 
unbiased outsider? We make the assumption that this type of personal 
attention is positive, however, what happens when personality conflicts 
prohibit communication? Might it be better that no performance development 
and review occur? 

Staff Development Through Graduate Study 

A number of studies relate to the importance of graduate study to 
successful Extension work. It appears to help successful performance. 
On the other hand, how does graduate study relate to a specific individual 
within their work environment. Is graduate study important if inservice 
education and inservice training are provided in ample quality and 
quantity? What input should the Extension organizations put into the 
development and design of graduate courses? Do Extension education courses 
really benefit agricultural or home economics county agents? Does 
performance improve as the percentage of staff in the state with advanced 
degrees increases? We make some large assumptions without supportive 
data. 

Performance Evaluation 

Performance evaluation is the process whereby a judgment is made about 
an individual's performance based on established goals, past performance 
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Or comparison with other similar staff members' performance. A basic 
reason for performance evaluation is to make a judgment and provide 
feedback in order to stimulate improved performance in the future and 
to provide a basis for merit salary increases or academic promotion* 
We really still do not know what successful performance of county Extension 
agents looks like. We do not know the relationship between performance 
evaluation as a process and an agent's job performance. We wrestle with 
the weighting of relative criteria and standards necessary to make a 
judgment on performance. We discuss the "correct" process for doing 
performance evaluation while wondering if there is such a process. 
If it would truly stimulate personnel development, this one process 
of performance evaluation needs a great deal more research to measure 
its impact on the effectiveness of an Extension organization. 

III. Conclusion 

The above questions and suggestions relate to a personnel development 
model which recognizes that an Extension faculty member exists in a work 
environment. The personnel development processes are repeated annually 
once the individual joins the organization. We all take our piece of the 
process as Extension administrators in an attempt to shape and mold the work 
environment in a better way. Yet most Extension Services have no master 
plan for personnel development. We accomplish the functions with a great 
deal of fervor—somewhat similar to a group of carpenters nailing boards 
with great enthusiasm, each following their own set of blueprints. Research 
on these functions and their relationships, and the impact they have on 
the individual staff member in Extension may be the most productive 





effort we as Extension workers can do to help our organization survive 
and succeed. Most importantly, we who are in personnel development need 
these answers to provide the best and most systematic environment for 
our colleagues' personal growth and successful Extension work performance. 
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Figure 1 
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INTRODUCTION 

A call for the identification of evaluation research needs 
for Extension education has come at a most opportune time. In 
fact, it closely follows a communication from the Administrator 
of Extension Service, USDA alerting State Extension Directors and 
others that Extension is increasingly being challenged to 
identify and evaluate program results, impacts and consequences. 
The communication further points out that questions asked of 
Extension in recent years, especially by Congress, have shifted 
from "how much and what are you doing with whom?" to "who is 
benefitting, by how much, and what difference did it make that 
Extension was involved?" (Greenwood, 1982). 

On first impression, it would seem that the establishment of 
program results, impacts and consequences would not be difficult. 
After all, Cooperative Extension Service does have a long history 
of "helping people to help themselves." Unfortunately, County 
Extension agent success in demonstrating program accomplishments 
through evaluation have been somewhat limited. Not only have 
many Extension evaluations encountered difficulties in measuring 
consequences, the results of some studies often have been 
misinterpreted. The Citizens Review Panel (1980), responsible 
for critiquing the national evaluation of Extension programs, 
expressed a deep concern that due to insufficient and sometimes 
misleading data, Congress could draw the wrong conclusions about 
the usefulness of Extension programs. 
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One explanation- for this limited success in evaluating 

Extension programs is offered by Patton (1980a) who says that the 

reason for these limitations is simple — there is no single 

treatment to be studied. Because every county and every state 

designs subject-matter educational programs differently, there is 

no such thing as "the" Extension program. Given the diffused 

nature of Extension programs, it is Patton's belief that national 

evaluations of national programs typically disguise more than 

they reveal. He explains (1980a: 73): 

When a standardized data-collection format is imposed 
upon thousands of local projects across the country and 
data collected in this standardized fashion is 
aggregated across the country, the result is more myth 
than reality . . . When national programs are 
implemented at the local level, the variations that 
result are so complex and so tied to local 
circumstances that no standardized format and no scheme 
of aggregation can reasonably represent and do justice 
to that complexity and those multitudes of adaptations, 

A second explanation for the limited success in measuring 
the effectiveness of Cooperative Extension Service is that most 
evaluation efforts have been limited in scope. Some have focused 
on standardized subjective measures of citizen reactions to 
Extension programs (Bennett, 1982; Christenson and Warner, 1982). 
Others have turned to effectiveness in meeting clientele needs 
including learning by clientele, applications of learning 
resulting in practices or actions carried out by clientele, and 
results of applications including social or economic benefits 
accruing to clientele (Kappa Systems, Inc., 1979). 

While such efforts may prove to be useful, generally, they 
have not provided adequate information to answer the question, 
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•'what difference did it make that Extension was involved?" Nor 
do they reflect the many ways that Extension is involved in 
helping people to help themselves. Hall (1982) argues that many 
of these activities are contradictory in nature. That is, 
organizations (such as Extension) may be more or less effective 
in regard to the variety of goals they pursue, the variety of 
resources they attempt to acquire, the variety of constituents 
inside and outside the organization and the variety of time 
frames by which effectiveness is judged. Hall (1982) concludes 
that no organization is effective. Instead, organization can be 
viewed as effective (or ineffective) to some degree in terms of 
its efforts to meet multipiv and conflicting environmental 
constraints, goals, constituents, and time frames. 

A third explanation for the limited success in measuring the 
effectiveness of Cooperative Extension Service is that while 
recognition has been given to the diffused nature of Extension 
programs, few have attempted to link standardized measures of 
individual performance to measures of program accomplishments. 
Such linkage is becoming increasingly necessary because, while 
role expectations of individuals may be determined within the 
broader organizational context (Kahn et al., 1964), there are 
times when individual discretions are crucial for organizational 
survival (Hall, 1982). In addition, multiple and often 
conflicting time frames and goals can have a major impact on both 
job satisfaction and productivity of individual employees (see 
Mulford et al., 1977). 
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Finally, most training efforts to improve the quality of 
program evaluations have focused primarily on improving the 
competencies of Extension field and support staff in evaluating 
individual programs. Only recently have efforts been directed 
toward the development of an evaluation system or framework by 
which evaluation efforts could be guided. Organizations that 
employ a systems approach to evaluation generally attempt to: 

1 . Evaluate the program as a whole. 

2. Use a problem solving approach with a s trong- 
relationship between evaluation and decision-making 
( ident if y relevan t evaluat ion questions for which 
decision-makers need information). 

3. Reach agreement between decision-maker and evaluator on 
data needed and how data will be used to answer the 
evaluation questions. 

Patton (1980b) describes this approach as evaluation 

research and offers the following definition: 

Systematic collection, analysis, and interpretation of 
information about the activities and outcomes of actual 
programs in order for interested persons to make 
judgements about specific aspects of what the program 
is doing and affecting. 

According to Patton (1980b) and many others, the purpose of 

evaluation research is to provide relevant and useful information 

to decision-makers, the criteria for usefulness and relevance 

being negotiated between evaluator and decision-maker during the 

conceptual phase of the evaluation. 

It is the position of the paper that a systems approach 

utilizing evaluation research methods is essential if Extension 

Service is to accurately determine "what difference did it make 

that Extension was involved?" It should be noted that neither a 
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systems approach nor the use of evaluation research methods are 
new to Cooperative Extension Service. Darcie Byrn and others 
(1959) have long advocated the use of the scientific approach in 
providing facts as a basis for making decisions, drawing 
conclusions, or forming judgements about the organization and 
conduct of extension work. They point out that v/hile evaluation 
may range from casual every day evaluation to scientific 
research, the difference in evaluation methods is a matter of 
degree rather than kind. It lies in the difference in the degree 
to which scientific method is necessary to the solution of 
problems. 

In today's environment of complex problems, budget 
restraints, and expensive program alternatives, a systems 
approach to evaluation can help resolve many of the factors 
hindering the successful implementation of evaluation efforts. 
It can also provide the foundation for training field, support, 
and administrative staff in measuring the ogani2a tional 
effectiveness of Cooperative Extension Service at the county, 
scate and national level. 

This paper will attempt to identify some of the evaluation 
research needs which must be addressed if Cooperative Extension 
Service is to base its evaluation research efforts on a systems 
perspective. Because the evaluation research needs of those at 
the county level often differ from the needs of those at the 
state or national level, both will be addressed. 
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EXTENSION EDUCATIONAL SYSTEM 

The first evaluation research need to be resolved is the 
diffused nature of Extension programs. If Pat ton's (1 980a) 
assessment is correct (there is no one Extension program), then 
it will be most difficult to use evaluation research technology 
in determining "what difference did it make that Extension was 
involved? " If programs are that varied , then state level 
Extension reports have little meaning. In addition, county level 
reports are not comparable. On the other hand, if there are 
major activities necessary for Extension to carry out its 
mission, these activities can provide the foundation upon which a 
systems approach to evaluation can be built. 

It may be recalled that the Extension Committee on 
Organization and Policy (ECOP) has endorsed the view of 
Cooperative Extension Service as a dynamic educational system 
oriented to the development of educational programs designed to 
meet the changing needs of diverse publics. ECOP believes that 
the strength of this system is in its involvement of people in 
the program development process in determ ining , planning and 
carrying out programs that meet their needs. In developing 
educational programs, Extension considers it especially important 
to involve people in identifying needs, concerns, and interests 
and to analyze problems that concern and affect them (Lawrence et 
al., 1 974). An illustration of the Extension educational system 
is provided in Figure 1 . 

(Figure 1 about here) 



38 



9 

ERIC 



Several major activities have been identified by Lawrence et 
al. (1974) as being critical to the effective implementation of 
the extension educational system. First, it is necessary for 
Extension staff to establish the program development concept of 
"helping people to help themselves. " The process of Extension 
education involves working "with" people and not "for 11 them. 
The second major activity is to involve people in the process of 
developing programs. This process requires interaction of 
clientele, Extension staff (including agents, specialists , 
supervisors, and administrators), and Extension advisory and 
support groups. Third, educational program priorities must be 
established. Lawrence et al. (1974) list four activities to help 
determine such priorities: 

1. Determine expressed needs of people as derived from 
local Extension program development committees. 

2. Analysis of the environment and other conditions of 
society. 

3. Emerging research results. 

4. Administrative response to recommendations and pressures 
of Cooperative Extension support groups. 

Program priorities are then reflected in an Extension plan of 
work. 

The fourth major program development activity of Cooperative 
Extension Service is to implement its educational programs as 
defined in a plan of work. Activities in support of this effort 
include arranging facilities and resources for educational 
programs, conducting teaching activities, coordination of 
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educational events with other organizations and involving lay 
people in the conduct of the Extension program. 

The final activity in the program development process is to 
objectively evaluate the progress and accomplishments made toward 
program goals. Extension staff are expected to assess the degree 
to which goals are achieved, determine the effectiveness of 
specific inputs for achieving change and use results of 
evaluations in redirecting program efforts and adjusting strategy 
(Lawrence et al., 1 974). 

This paper advocates using the major activities of the 
Extension program development process as the foundation for 
evaluating the effectiveness of Cooperative Extension Service. 
These activities are equally applicable at the county, state, and 
national level. In addition, the scope of an evaluation research 
effort could be as narrow as a specific subject-matter goal at 
the county' level or as broad as an interdisciplinary goal at the 
state or national level. The important point is that each of the 
program development activities can be evaluated in a systematic 
fashion. If the strength of Cooperative Extension is in its 
involvement of people, then it is just as important to measure 
the consequences of people's involvement in developing 
educational programs as the consequences of their part icipating 
in educational programs. 

Several evaluation research questions that can be used in 
determining consequences of people's involvement in developing 
educational programs are as follows: 
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1. What is the problem? (Nature, extent and location of 
specific problem.) 

2. Who is the target population? 

3. How doe* the goal and list of activities of the 
educational program relate to the specific problem of 
the target audience? 

4. Can the delivery system des igned reach the target 
population effectively? 

5. How does helping the target audience to solve the 
identified problem support the mission of the 
organization? 

It also is important to measure the implementation activity 
of the program development process. The utility of such 
measurement is aptly illustrated by Rossi et al. (1979: 132) who 
have found that a large proportion of programs that fail to show 
impacts are really failures to deliver interventions (programs) 
in ways specified. The authors list three potential failures: 

1. Little or no program is deliverable to the clientele. 

2. The wrong program is delivered. 

3. The program is uncontrolled, uns tandardized or varies 
across target populations. 

To reduce the possibility of such failures, the following 

research evaluation questions should be addressed: 

1. What types of educational activities and subject matter 
content were offered? 

2. Who participated in the program? 

a. Characteristics of participants and nonparticipants 

b. Intensity of participation by different client 
groups 

3. What are the reactions of participants and others to the 
quality and content of the educational program? 
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In summary, while the Congress of the United States and 
others may have shifted the emphasis of their questions to "who 
is benefitting, by how much, and what difference did it make that 
Extension was involved, 11 it must first be established "why" and 
"how" Extension was involved* Then it can be determined if a 
planned program designed to produce intended changes in a target 
population achieved its objective* 

EVALUATION RESEARCH DESIGNS 

A second evaluation research need relates to the types of 
evaluation research designs chat should be used to determine 
"what difference did it make that Extension was involved." It 
may be recalled that Darcie Byrn and others (1959) have 
illustrated evaluation approaches along a continuum ranging from 
informal (casual everyday evaluations, self -checking evaluations, 
and do it yourself surveys) to formal (Extension studies and 
scientific research). Unfortunately for county Extension agents, 
it seems that while informal evaluations are easier to implement, 
their results often are difficult to interpret. Conversely, 
while formal evaluations could provide interpretable results, 
they often are time-consuming and difficult to implement. 

This section of the manuscript will discuss the types of 
changes made by target audiences that are most often measured and 
the evaluation designs necessary to measure such changes. In 
addition, criteria for determining appropriate evaluation designs 
will be provided. 
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Measurement of Change 
There are several types of changes that could be achieved by 
a target population. As such, an evaluation research effort 
could be directed toward collecting data to answer one or more of 
the following questions. 

1 . Did a change occur in the target population? 

a. Attitudes changed 

b . Knowledge acquired 

c. Skills developed 

d. Practices adopted 

2. What are the consequences of changes made? 

3. What is the cause of the change (or lack of change)? 

Evaluation Designs 
Two types of evaluation designs are available for use in 
determining whether a change occurred in the target audience: 
designs that estimate change and designs that measure change and 
attempt to explain the cause of the change. Designs commonly 
used to estimate change include: 

1. Projections based on other studies 

2. Projections from clientele 

3. Follow-up survey of participants 

4. Case study 

5. Observations by experts 

For some decision-makers, an estimate of changes made by 
clientele is satisfactory. For others, an accurate measure of 
change and the cause of the change is important. Possible 
explanations for causes of change that have been identified 
(Campbell and Stanley, 1966) include the following: 
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1. 



Some would have made the change anyway. 



2. Long-term trends mask program accomplishments 
(inflation) . 

3. Shore-term events influence accomplishments (weather). 

4. Those who participate in Extension programs are 
volunteers. As such they possess personal 
characteristics which may differ from nonpart icipants. 

5. The program influenced the participants to make changes. 
In evaluat ions where change is measured by estimate, 

interpretations of the results are difficult and often 
unconvincing (Fitz-Gibbon and Morris, 1978). When more accurate 
measures are needed by decision-makers, comparison groups may be 
necessary. The following evaluation designs are recommended when 
comparison groups are required (Rossi et al., 1979): 

1. Randomized design. Every person in the target audience 
has the same chance as any other person to be selected 
to participate either in the program or in a control 
group that does not receive the same program. In other 
words, different groups receive different programs or 
receive the same programs on a delayed basis. 

2. Constructed controls. Persons who do not participate in 
the educational program but who are similar to those who 
do participate are compared to the participants. 
Control groups may be established through individual or 
aggregate matching. Individual matching, generally, is 
preferred . 

3. Statist ic a 1 controls. Use of multivariate analysis 
techniques to measure accomplishments of programs that 
have been in operation for several years. The 
reliability of the measure is established statistically. 
Cross sectional studies often are used to provide such 
measures. 

4. Reflexive controls. Many Extension programs contain 
long-range goals and are carried out with the same 
clientele for several years. Such programs allow 
participants to serve as their own reflexive controls. 
The procedure, referred to as time-series design, 
requires that individuals be measured for at least three 



ERIC 



44 



points in time* It is recommended, but not required, 
that the first measurement point be before or early in 
the implementation phase* 

In considering the four designs, it should be noted that 
they are neither easy to implement nor without limitations* For 
those Extension programs where it is truly important for 
Extension Service to provide, with as much certainty as possible, 
measurement of change and causes of changes made by those who 
have participated, such evaluation designs may be necessary* 
However, unless the evaluator has some knowledge of the 
strengths, limitations, and appropriateness of each design; the 
evaluation results could provide insufficient or misleading 
information and, thereby, influence others to draw wrong 
conclusions about the effectiveness of Extension programs. 

Current efforts by Extension Service, USDA in calling for 
State Extension Services to provide accomplishment reports and 
impact evaluations appear to be a very positive step in helping 
states determine appropriate evaluation designs (see Greenwood, 
1982). Accomplishment reports consist of brief narrative reports 
of program results representing statewide aggregation of data. 
The intent of an accomplishment report appears to be one of 
estimating changes made by target audiences. Conversely, impact 
evaluations are technically valid in-depth national or state 
evaluations conducted to assess: 

1. The economic or social consequences of selected high 
priority programs. 

2. Specific other aspects of Extension inputs, operations,, 
or programs. 
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EVALUATION REPORTS 

A third major evaluation research need relates to the 
reporting of evaluation results. One of the most discouraging 
facets of evaluation research is when the evaluation report is 
not read by the audience for whom it was prepared. The 
procedures recommended in this paper do not insure that the 
report will be read or that the results used. It does insure 
that the decision-maker will be aware that the evaluation is 
being conducted and that its purpose is to provide the decision- 
maker with relevant and useful information about what the 
educational program is doing and affecting. 

The following outline is provided as a guide in the 
development of an evaluation report (Wentling, 1980). This 
outline serves a second purpose in that it identifies evaluation 
research needs that must be addressed in evaluating educational 
programs. 

I. Evaluation Objectives 

A. Purpose of evaluation 

B. Audience of the report 

II. Program Description 

A. Statement of problem 

B. Program goals 

C. Characteristics of participants (number and type) 

D. Description of educational activities conducted and 
subject matter covered 

III. Description of Evaluation Methodology 

A. Specific parts of the program development process to 
be measured 

B. Time frame of evaluation 

C. Description of evaluation procedures 

D. Possible bias of the methods used 
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IV. Results 

A. Reactions of target audiences 

B. Changes made by target audience 

V. Evaluation Conclusions 

A. Consequences of the changes made 

B. Effectiveness of the Extension program 

C. Recommendations 

PERFORMANCE APPRAISAL 
The final evaluation research need to be discussed in this 
paper refers to the linkage between performance appraisal and 
program accomplishments. At this time, a relationship has not 
been adequately established between measures of individual 
behavior necessary for effective performance (inputs), individual 
performance (outputs) and changes in behavior of constituency 
(program accomplishments). It is the position of this paper that 
the program development process can be used to establish such a 
relationship. An illustration of this relationship is presented 
in Figure 2. 

(Figure 2 about here) 
An accurate and relevant performance appraisal system could 
contribute to determining whether programs are being carried out 
in ways specified in the plan of work. However, most Extension 
performance appraisal methods have been based on personal traits 
or characteristics having little relationship to measurement of 
organizational effectiveness. In recent years, there has been 
growing support by Extension administrators and others for an 
accurate and relevant performance appraisal system that could 
contribute not only to judgemental decisions about personnel but 



also to development decisions concerning individual performance 
related to program accomplishments. 

According to Cumm ings and Schwab (1973), judgemental 
appraisal provides results for administrative dec is ions 
concerning such aspects as rewards based on merit, promotions, 
transfer and termination. Developmental appraisal provides 
results to help improve performance by identifying areas for 
program improvement and individual growth. By focusing on both 
past accomplishments and on improvements in future performance, a 
performance appraisal system becomes an important component of 
organizational effectiveness. 

Methods of Performance Appraisal 
Two general approaches to performance appraisal can be 
linked to organizational effectiveness. One approach to 
performance appraisal focuses on behavior by identifying the 
critical behaviors necessary for effective performance. A second 
approach concentrates on measurement of accomplishment of 
performance goals (Haynes, 1978). 

Critical Behavior 
Performance appraisal methods used in measurement of 
critical behavior necessary for effective performance generally 
are based on comparisons of individuals to absolute standards. 
Procedures used to compare individual performance to absolute 
standards normally follows one of two approaches. The first 
approach uses a qualitative basis to determine the existence of 
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performance behavior. That is, the supervisor is asked to 
indicate whether the employee possesses or does not possess the 
performance characteristics of interest. The second approach 
attempts to quantify the degree to which the employee possesses 
certain performance characteristics. 

With some modifications, comparison of individuals to 
standards can be related directly to measures of organization 
effectiveness. The following sections describe the nature of 
those modifications. 

Existence of Behavior 

The development of indicators to measure the existence of 
performance characteristics generally is based on the critical 
incidents technique. The critical incident technique identifies 
activities that are critical to one's job. Outcomes of 
performance are then measured against those activities. The 
advantage of the critical incident technique is that it provides 
more feedback of required information and it can seek available 
items that have been shown to differentiate between successful 
and unsuccessful performers. The disadvantage of the critical 
incident approach is that considerable time is needed for 
collecting items critical to one's job performance. A partial 
listing of critical incidents in program development was reported 
in Figure 2. 

The weighted checklist method generally is used to measure 
behavioral outcomes of performance using a conventional interval- 
level scale of measurement. An example weighted checklist is 
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presented in Figure 3. One's performance evaluation score is 
determined by summing the weights of the outcomes that have been 
checked (Curamings and Schwab, 1973). As illustrated by the 
outcomes listed in Figure 3, critical incidents necessary to 
successful job performance also are necessary for effective 
programs. 

(Figure 3 about here) 
Knauft (1948) describes four advantages in using the 
weighted checklist to score outcomes of performance. 

1. It provides an objective and rapid method of scoring the 
completed forms. 

2. The exact values or weights of each outcome are unknown 
to the raters and may not be readily deduced by them. 

3. Raters require very few directions and no extensive 
training program in rating techniques. 

4. The weighted checklist may be used as an overall 
criterion measure if it is constructed to sample a large 
number of different aspects of the employee^ on-the-job 
behavior. 

The weak point of the weighted checklist is arriving at a proper 
weighting of various items on the checklist (Haynes, 1978). 

Degree of Behavior 

Linkage between organizational effectiveness and the degree 
to which employees display critical behavior necessary for 
effective performance can be obtained using a behaviorally- 
anchored rating scale. According to Cummings and Schwab (1973), 
scale items are based on the critical incident process described 
earlier and are divided into several categories of items. The 
value assigned to each item is weighted (similar to the weighted 
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checklist) from very low to very high* The evaluator is asked to 
select the one item which best describes the employee's behavior 
for each category measured* An employee's evaluation score is 
the sum of the category scores. An example behaviorally anchored 
rating scale is presented in Figure 4, 

(Figure 4 about here) 

Performance Goals 
The second major approach to performance appraisal relates 
to accomplishment of performance goals. Measurement of 
performance goals involves the supervisor and the employee in a 
discussion to set performance goals or objectives in support of 
the organization. Performance appraisal methods often used in 
such settings include management by objectives and direct 
indexes. 

Management by Objectives 

Management by objectives (MBO) measures performance in terms 
of goal accomplishment. According to Cummings and Schwab (1973), 
the employee (with supervisor approval) defines goals to be 
accomplished during some specified time period. Goal statements 
may be set for methods, means of accomplishment, or desired 
outcomes.^" MBO can be related directly to organizational 
effectiveness when such goals are stated in terms of desired 



Goal statements related to methods or means of accomplishment 
are often based on standards of critical behavior, 
determinant of performance. In addition, performance can be 
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program outcomes. Illustrations performance goals were 
reported in Figure 2. 

One major strength of MBO is that it sets unique goals for 
each employee and supports the view that goals are an important 
accurately measured in situations where jobs can vary with the 
capabilities and interests of the individual. Conversely, MBO 
can lead to the unrealistic adoption of rigid and/or quota- 
oriented goals. Also, MBO provides limited information for the 
equitable distribution of rewards because goals are 
individualized and may not be comparable across individuals, 
hhen such comparisons are needed, measures of critical behavior 
should be used. 

Direct Indexes 

A second measure of goal accomplishment measures 
productivity directly. Direct indexes are used to measure 
quantity or quality of output. Examples of quantitative 
indicators include hours worked, miles traveled, meetings 
conducted, publications prepared, people reached, and changes 
made by clientele. Quality measures include clientele reactions 
to programs and publications and supervisory observation of 
agents in action. As reported earlier, Christenson and Earner 
(1982) describe how constituent satisfaction can be used as a 
measure of organizational effectiveness. 
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Summary 

This section has attempted to link critical behaviors 
necessary for effect ive performance (inputs) individual 
performance (outputs) and changes in behavior of constituency 
(program accomplishments). This linkage is becoming increasingly 
necessary as county Extension agents and others attempt to cope 
with multiple and other conflicting environmental constraints, 
goals, constituencies, and time frames. 

The primary evaluation research needs to be met if such 
linkages are to occur is the validation of the performance 
appraisal system. That is, the procedures used to construct and 
test a performance appraisal instrument must be verifiable. 
Steps in the development of a performance appraisal instrument to 
be verified include job analysis (content analysis), item 
selection, item scoring system followed, and procedures used to 
measure reliability and validity. 

The Ohio Cooperative Extension Service is in an excellent 
position to establish the linkage between individual and program 
performance because it has already undertaken a preliminary test 
to verify its performance appraisal system (Ladewig and Shiao, 
1983). A remaining task is to more closely tie performance 
appraisal to an overall system of organizational effectiveness. 

SUMMARY AND CONCLUSIONS 
There is much concern that the diffused nature of Extension 
programs restricts the ability of Cooperative Extension Service 
to meaningfully evaluate its programs. This manuscript argues, 
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however, that Extension does have an educational program that can 
be evaluated — the program development process. In fact, 
Extension leaders have stated that the involvement of citizens in 
the development and implementation of educational programs is one 
of the major strengths of the Extension educational system in 
carrying out its mission of "helping people to help themselves." 

Because the major activities of Cooperative Extension 
Service relate closely to the involvement of the target audience 
in the program development process, this paper has recommended 
that efforts to evaluate the organizational effectiveness of 
Cooperative Extension Service should be based on the Extension 
educational system of program development. It was pointed out 
that because all organizations are effective to some degree in 
terms of specific constraints, goals, constituents, and time 
frames, the contradictory nature of organizational effectiveness 
must be considered in analyzing effectiveness of Extension 
Service to carry out its system activities. It was recommended 
that because of the importance of individual performance in 
carrying out such activities, measures of program accomplishments 
should be linked to measures of individual performance. 

In sum, this manuscript has advocated that a systems 
approach utilizing evaluation research technology is essential if 
Cooperative Extension Service is to accurately determine "what 
difference did it make that Extension was involved. " Neither the 
systems approach nor the evaluation research methods advocated 
are new to Cooperative Extension Service. What is different is 
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the degree to which the scientific method is necessary in 
providing facts as a basis for making decisions, drawing 
conclusions, or forming judgements about the organization and 
conduct of extension work. 

Five procedural requirements are necessary if such an 
approach is to provide accurate and relevant results. First, 
evidence of relevant problems to be addressed must be 
established. Without documentation of constituent problems, goal 
accomplishments may have little meaning. Second, citizen 
involvement in program development must be documented. This will 
enable analysts and others to recognize both the range of options 
open to Extension Service and the existence of factors beyond 
organizational control as Extension copes with constraints and 
mandates in working toward goal accomplishments in support of its 
mission. Third, a job analysis of critical incidents necessary 
for effective performance must be accomplished to provide a basis 
from which defensible performance standards can be derived. 
Otherwise, it will be most difficult to link measures of 
performance appraisal to measures of program accomplishments. 
Fourth, the information needs of decision-makers within and 
external to Cooperative Extension Service must be considered. 
Finally, the resources necessary to carry out recommended 
evaluation research design must be provided if Extension is to 
accurately answer M what difference did it make that Extension was 
involved? " 
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Figure 1 
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Figure 2 



Personnel and Program Evaluation of the 
Cooperative Extension Educational System 



System Component 



Establish support 
for program 
development 

Implement program 
development 
process 



Program 
determination 



Program 
implementation 



Performance Evaluation 
County Extension Agents 



Program 
evaluation 



Critical Behavior 
( Inputs)* 

Interact with 
constituents 



Identify potential 
members for 
program develop- 
ment committee 



Plan meetings 
Provide resources 



Advertise programs 
using relevant 
dissemination 
methods 

Prepare for educa- 
tional programs 



Develop an 
evaluation plan 



Performance Goals 
(Outputs)* 

Cons t ituen ts accept 
concept of program 
development 

Program development 
comm i t tee organ - 
ized and trained 
in program 
development 
process 

Plan of work 



Conduct timely 
educational pro- 
grams on relevant 
topics us ing 
appropriate 
educational 
techniques 

Conduct program 
evaluation and 
prepare a report 
of findings 



Program Evaluation 

Constituent Behavior Desired 
(Results)* 

Level and quality of support 
for program development 



Constituency representa- 
tiveness of the committee 

Knowledge of the 
committee 



Committee involvement in 
planning programs 

Documentation of relevant 
problems and target 
audiences to be addressed 

Involvement of target 
audiences 
Reactions of target audiences 



Measure changes made by 

target audiences 
Use evaluation report for 

program accountability and 

program improvement 



* Examples provided as illustrations 



Figure 3 

Example Weighted Checklist of Behavioral Outcomes 



Scale 

Item Values 

Has good community representation at planning sessions 3.5 
to set county priorities 

Conducts well-organized educational programs 1.5 

Minority involvement in program exceeds minimum 3.5 
requirements 

Programs branch out to reach new audiences 1.5 

Has little or no premature turn over among 3.5 
volunteer leaders 

Uses needs assessment survey results to determine 2.5 
relevant program priorities 



Source: American Institutes for Research, 1979. Manual for 
County Extension Agents --Performance Review, Analysis, and 
Planning. Washington, D.C. 
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Figure 4 



Example Behaviorally Anchored Rating Scale of 
Program Implementation Behaviors* 



Category 



Value 



Item 



Extremely good 
performance 



Very good 
performance 



Good 

performance 



Neither poor 
nor good 
performance 

Slightly poor 
performance 

Poor 

performance 



Extremely poor 
performance 



Conducts programs in the county in all 
locations to meet the needs of all 
people, irrespective of race, color, 
national origin, or economic 
c ircumstances. 

Always does some teaching in every 
educational activity for which he/she has 
responsibility. 

Spec ial ist involvement in educational 
programs using appropriate methods of 
teaching . 



Involves a few volunteers and 
leaders in implementing programs. 



local 



3 Gives generalized answers to callers with 
specific inquiries. 

2 When he/she receives calls for technical 
assistance, make? excuses for not 
providing assistance requested. 

1 Gives information not documented by 
research . 



*Items are provided strictly for illustrative purposes. Values 
are not based on empirical evidence. 



Source: American Institutes for Research, 1979. Manual for 
County Ext ens ion Agent s --Performance , Rev iew , Analys is and 
Planning. Washington, D.C. 
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PARTICIPANTS 



MAY 25 & 26, 1983 



GRADUATE STUDENTS 

Room 208, Ag. Admin* Bldg. 
2120 Pyffe Road 
Columbus, OH 43210 
(614) 422-6321 

Kodran Al-Zahrani 
Aruk Amba 

Marcia Andrus 
Bill Beckley 
Karen Behm 
Debbie Bragg 
Cheryl Caplinger 
Lu Etuk 
Martha Fix 
Debbie Flasher 
A. Oto-France 



Kathleen A. Gutheil 
Lisa Holmes 
Bob Horton 
Chris O. Igodan 
Jacob Ilkiuyoni 
Barry Jolliff 
Sue Kohls 
Kamiar Kouzekanani 
Tom Krill 
Kristin Meyers 
Juanita Miller 



Ken Milliser 
Esmawi B. Othman 
Pantelis Ritsos 
Stephen Schumacher 
Mohammed Shibah 
Okon Uko 
Judith Waldman 
George Wardlow 
Larae Watkins 
Vicki Whitted 



AGRICULTURAL EDUCATION FACULTY 

Kirby Barrick, 2120 Fyffe Road, Columbus, OH 43210, (614) 422-6321 

Stan Burke, ATI Wooster, Wooster, OH 44691, (216) 262-5956 

Gil Guiler, 2120 Pyffe Road, Columbus, OH 43210, (614) 422-0202 

William Hull, 1960 Kenny Road, Columbus, OH 43210, (614) 486-3655 

Jim Knight, 2120 Pyffe Road, Columbus, OH 43210, (614) 422-6321 

J.D. Mccracken, 2120 Pyffe Road, Columbus, OH 43210, (614) 422-0202 

Larry Miller, 2120 Fyffe Road, Columbus, OH 43210, (614) 422-6671 

L.H. Newcomb, 2120 Fyffe Road, Columbus, OH 43210, (614) 422-6321 

Robert Norton, 1960 Kenny Road, Columbus, OH 43210, (614) 486-3655 

J. Robert Warmbrod, Chairman, Ag. Ed., 2120 Fyffe Road, Columbus, OH 43210, 
(614) 422-6321 

COOPERATIVE EXTENSION SERVICE (State) 



Clarence Cunningham, Assistant Director, Administration, 2120 Fyffe Road, 
Columbus, OH 43210, (614) 422-4067 

Marcus Dresbach, Leader, Energy, 2120 Pyffe Road, Columbus, OH 43210, 
(614) 422-8436 
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William Faber, Extension Floriculturist, 2001 Fyffe Ct., Columbus, OH 43210 
(614) 423-8200, 422-9784, 422-9027 

Loren Geistfeld, Chairman, Home Management and Housing, 1787 Neil Ave., 
Columbus, OH 43210, (614) 422-4389 

George Gist, Acting Director, 2120 Fyffe toad, Columbus, OH 43210, 
(614) 422-4067 

James R* Helt, Associate State Leader, 4-H, 2120 Fyffe Road, Columbus, OH 
43210, (614) 422-4444 

David Jenkins, Leader, Professional Improvement, 2120 Fyffe Road, Columbus, 
OH 43210, (614) 422-5178 

Ted Jones, Assistant Director, Agricultural Industry, 2120 Fyffe Road 
Columbus, OH 43210, (614) 422-4077 

Charles Lifer, Assistant Director, 4-H, 2120 Fyffe Road, Columbus, OH 
43210, (614) 422-4444 

Linda Pursley, 4-H Specialist, Literature and Curriculum Dev., 2120 Fyffe 
Road, Columbus, OH 43210, (614) 422-4444 

Linda Roberts, Program Coordinator, Home Economics, 1787 Neil Ave., Columbus 
OH 43210, (614) 422-4481 

John Rohrer, Leader, Community Service, 2120 Fyffe Road, Columbus, OH 43210 
(614) 422-8436 

Alma Saddam, Nutrition Specialist, 1787 Neil Ave., Columbus, OH 43210, 
(614) 422-0827 

Keith Smith, Leader, Personnel Development, 2120 Fyffe Road, Columbus, OH 
43210, (614) 422-6181 

John Stitzlein, Leader, Personnel, 2120 Fyffe Road, Columbus, OH 43210, 
(614) 422-6181 

Paul R. Thomas , Assistant Director, Community and Natural Resource 
Developmenmt, 2120 Fyffe Road, Columbus, OH 43210, (614) 422-8436 

David Tipton, Extension Editor, 4-H, 2021 Coffey Road, Columbus, OH 43210, 
(614) 422-2011 

Judy Weasel, Management and Equipment specialist, 1787 Neil Ave., Columbus, 
OH 43210, (614) 422-4918 

COOPERATIVE EXTENSION SERVICE (Field) 

Gilbert Atkinson, Area Supervisor, 10 West Auglaize St., Wapakoneta, OH 
45895, (419) 738-8197 

David Boothe, Area Supervisor, 17 Standpipe Rd., Jackson, OH 45640, 
(614) 286-2177 
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Lydia Bruny, Co Ext Agt, HE, 104 S, Columbus, Somerset, OH 43783, 
(614) 743-1602, 743-1406 



Sam Crawford, Area Ext Agt, CNRD, 17 Standpipe Road, Columbus, OH 45640, 
(614)286-2177 

Herbert Crown, Area Supervisor, 80 N. Walnut Street, Mt. Gilead, OH 43338, 
(419) 946-8015 

Bobbie Davis, Co Ext Agt, Agr & CNRD, 11100 Winton Rd., Greenhills, 
Cincinnati, OH 45218, (513) 825-6000 

William Henderson, Co Ext Agt, 4-H, 219 W. Northern, Lima, OH 45801, 
(419) 222-9946, 229-3736 

Jo Jones, Area Ext Agt, HE, 1394 E. Main St., Eaton, OH 45320, (513) 456-5508 

Marion Kroetz, Area Supervisor, North Clinton St., Route 2, Defiance, OH 
43512, (419) 523-6294 

Willie Larkin, Area Ext Agt, 4-H, 1394 E. Main St., Eaton, OH 45320, 
(513) 456-5508 



Mary At Murphy, Asst Agt, Youth EFNEP, 1001 S. Main St., Dayton, OH 45409, 
(513) 224-9654 

John Parker, Jr., Area Supervisor, 490 S. Broad St., Canfield, OH 44406, 
(216) 533-3453, 533-4727 

Joe Pittirtan, Area Supervisor, 16714 S.R. 215, Caldwell, OH 43724, 
(614) 732-2381 

Peggy Simmons, Co Ext Agt, HE, 1810 Princeton Rd., P.O. Box 958, Hamilton, OH 
45012, (513) 867-5925 

Ann Smith, Co Ext Agt, HE, Courthouse, Zanesville, OH 43701, (614) 454-0144 

John Ulmer, Co Ext Agt, 4-H and CRND, 104 S. Columbus, Somerset, OH 43783, 
(614) 743-1602, 743-1406 
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